Flicker correction method and device, and imaging device

ABSTRACT

Even when the light intensity of the light source varies, the flicker correction can thus be made flexibly. The present invention provides a flicker correction method comprising the steps of predicting, from an image of a present flicker-corrected frame, a flicker of an image of a next frame to generate two types of flicker images having flickers different in level from each other added thereto, detecting a flicker component through comparison between the generated two types of flicker images and an image of an input next frame, generating a flicker correction value on the basis of the detected flicker component, and making flicker correction by adding the generated flicker correction value to an input image frame by frame.

CROSS REFERENCES TO RELATED APPLICATIONS

The present invention contains subject matter related to Japanese PatentApplication JP 2005-141614 filed in the Japanese Patent Office on May13, 2005, the entire content of which is incorporated herein byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a flicker correction method and device,and an image pickup device, in which a flicker is corrected bysubtracting a flicker correction signal from an image signal.

2. Description of the Related Art

In the conventional image sensor, the timing of charge storage differsdepending upon whether the charge storage is made per plane or per line.Generally, timing of the charge storage per plane is called “globalshutter system” while timing of the charge storage per line is called“rolling shutter system”. Most of the CCDs have adopted an image sensorof the global shutter type in the past. Recently, however, increasinglymore attention has been paid to the CMOS image sensors that consume lesspower than the CCDs and can be produced more inexpensively than the CCDsbecause of their smaller number of parts. Many of the CMOS image sensorsadopt the rolling shutter system for their structural problem. With oneof the two shutter systems, when imaging is made under the light of alight source repeating turning on and off, light and dark fringes willappear in an entire image plane (plane flicker), while with the othershutter system, light and dark fringes will appear per line (in-planeflicker).

FIG. 1 shows a difference in amount of charge storage in an image sensoradopting the global shutter system, and FIG. 2 shows an example of imageincurring a plane flicker when the image sensor is of the global shuttertype. FIG. 3 shows a difference in amount of charge storage in an imagesensor adopting the rolling shutter system, and FIG. 4 shows an exampleof image incurring an in-plane flicker. Also, a flicker componentincluded in an image captured under the light of a light sourcecyclically turning on and off can be approximated to a sinusoidal wave,and there has been prevalent the method of forming a corrected image byremoving the flicker on the basis of the nature of the sinusoidal wave.

For the flicker correction, there was proposed a method of controllingthe gain on the basis of a flicker component detected in an input image(as in the Japanese Patent Application Laid Open No. 2004-222228.

SUMMARY OF THE INVENTION

When an object is imaged with a digital camera under the light of alight source that repeats turning on and off cyclically, such as afluorescent lamp, cyclic light and dark fringes will appear in acaptured image of the object, resulting in that they will seemingly runin the image. Otherwise, there will cyclically take place a differencein brightness between frames over an image. This is called “flicker”.The flicker is a problem unavoidably taking place when an object isimaged with a digital camera using an image sensor to image the objectunder the light of a flickering light source with the timing of chargestorage being shifted.

For flicker correction, the feature that a flicker can be approximatedto a sinusoidal wave is utilized to detect a flicker component in aninput signal. Similarly, a correction amount is calculated from thecharacteristic of the sinusoidal wave and detected flicker component,and the correction amount is added to the input signal or the latter ismultiplied by the correction amount. For approximation of the flickercomponent to the sinusoidal wave, three features “cycle”, “phase” and“amplitude” have to be detected.

A cycle can be detected based on a power supply frequency and framerate.

On this account, the Applicant of the present invention proposed aflicker correction method including the steps of acquiring a flickercorrection signal corresponding to a flicker component included in eachof specific periods of a video signal formed from a succession of thespecific periods and containing the flicker component in response to acorrection error signal for each specific period and calculating theflicker correction signal and each specific period to generate acorrected video signal for one specific period, whose flicker componenthas been corrected, detecting a correction error of the flickercomponent in the corrected video signal for one specific period beforeeach of the specific periods and each specific period to acquire thecorrection error signal as one corresponding to the detected correctionerror, and acquiring the flicker correction signal as one which reducesthe correction error correspondingly to the correction error signal (asin the Japanese Patent Application Laid Open No. 2004-330299).

The above flicker correction method is performed with no flickeramplitude being detected but with a fixed value of flicker. In case thelight from the light source does not vary, the method can be performedwithout any problem even with a fixed value of flicker amplitude.However, in case the light from the light source varies, the flickeramplitude has to be changed appropriately. This conventional method isthus not versatile.

It is therefore desirable to overcome the above-mentioned drawbacks ofthe related art by providing a flicker correction method and apparatusand an image pickup device, in which flicker can flexibly be correctedeven when the light intensity of a light source varies.

According to the present invention, flicker correction is made bydetecting a flicker component in an input image signal, calculating acorrection value based on the detected flicker component and adding thecorrection value to the input image signal.

According to the present invention, there is provided a flickercorrection method including the steps of:

predicting, from an image of a present flicker-corrected frame, aflicker of an image of a next frame to generate two types of flickerimages having flickers different in level from each other added thereto;

detecting a flicker component through comparison between the generatedtwo types of flicker images and an image of an input next frame;

generating a flicker correction value on the basis of the detectedflicker component; and

making flicker correction by adding the generated flicker correctionvalue to an input image frame by frame.

According to the present invention, there is also provided a flickercorrection device including:

a flicker correcting means for making flicker correction by adding aflicker correction signal to an input image signal; and

a flicker correction signal generating means for predicting a flicker ofan image of a next frame from the image signal having beenflicker-corrected by the flicker correcting means and an image signalnot yet flicker-corrected to generate two types of flicker images towhich flickers different in level from each other, detecting a flickercomponent through comparison between the generated two types of flickerimages and an image of an input next frame and generating a flickercorrection value on the basis of the detected flicker component,

the flicker correcting means adding, to the input image signal, theflicker correction signal generated frame by frame by the flickercorrection signal generating means to make flicker correction.

According to the present invention, there is also provided an imagepickup device including a flicker correction device to make flickercorrection by adding a flicker correction signal to an image signalcaptured by the image pickup device, the flicker correction deviceincluding:

a flicker correcting means for making flicker correction by adding aflicker correction signal to an input image signal; and

a flicker correction signal generating means for predicting a flicker ofan image of a next frame from the image signal having beenflicker-corrected by the flicker correcting means and an image signalnot yet flicker-corrected to generate two types of flicker images towhich flickers different in level from each other, detecting a flickercomponent through comparison between the generated two types of flickerimages and an image of an input next frame and generating a flickercorrection value on the basis of the detected flicker component,

the flicker correcting means adding, to the input image signal, theflicker correction signal generated frame by frame by the flickercorrection signal generating means to make flicker correction.

According to the present invention, the amplitude of flickers cansequentially be predicted through comparison between anamplitude-predicted flicker image and an image captured by an imagesensing device (will also be referred to as “imaging element” herein).Even when the light intensity of the light source varies, the flickeramplitude can flexibly be adjusted in an automatic manner to correct aflicker at any time. Also, even when imaging is made while relocatingfrom a place with a flicker source to a flicker-free place, the flickeramplitude can automatically be varied and the flicker correction beceased. Further, even when imaging is made while relocating aflicker-free place to a place with a flicker source, the flickeramplitude can automatically be varied and the flicker correction bemade.

Therefore, according to the present invention, even when the lightintensity of the light source varies, the flicker correction can thus bemade flexibly.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically illustrates a difference in amount of chargestorage in an image sensor of the global shutter type;

FIG. 2 schematically illustrates an example of plane flicker imageappearing when the global shutter system is adopted;

FIG. 3 schematically illustrates a difference in amount of chargestorage in an image sensor of the rolling shutter type;

FIG. 4 schematically illustrates an example of in-plane flicker image;

FIG. 5 is a schematic block diagram of an image pickup device as oneembodiment of the present invention;

FIG. 6 is also a schematic block diagram of a flicker correction circuitincluded in the image pickup device shown in FIG. 5;

FIG. 7 is a schematic block diagram of a correction value calculator inthe flicker correction circuit;

FIG. 8 schematically illustrates an algorithm for correction errordetection in the image pickup device;

FIG. 9 is a schematic block diagram of a correction phase error detectorincluded in the image pickup device;

FIG. 10 is a schematic block diagram of a correction amplitude errordetector included in the image pickup device;

FIG. 11 is a schematic block diagram of a flicker amplitude adjusterincluded in the image pickup device.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention will be described in detail below concerning theembodiments thereon with reference to the accompanying drawings. Itshould be noted that the present invention is not limited to theembodiments which will be described herebelow but it may be can bemodified in various manners, constructed alternatively or embodied invarious other forms without departing from the scope and spirit thereof.

The present invention is applicable to an image pickup deviceconstructed as shown in FIG. 5. The image pickup device is generallyindicated with a reference numeral 100.

The image pickup device 100 includes a red color image sensing device(image element) 10R, green color image sensing device (imaging element)10G, blue color image sensing device (imaging element) 10B, A-Dconverters 20R, 20G and 20B to digitize image signals SI_R, SI_G andSI_B of color images captured by the image sensing devices 10R, 10G and10B, respectively, flicker correction circuits 30R, 30G and 30B,correction phase error detectors 40R, 40G and 40B and correctionamplitude error detectors 50R, 50G and 50B, supplied with the imagesignals DV_R, DV-G and DV_B digitized by the A-D converters 20R, 20G and20B, respectively, flicker amplitude adjusters 60R, 60G and 60B suppliedwith correction amplitude error signals C_R, C_G and C_B detected by thecorrection amplitude error detectors 50R, 50G and 50B, respectively,camera signal processing circuit 70 supplied with image signals CV_R,CV_G and CV_B flicker-corrected by the flicker correction circuits 30R,30G and 30B, respectively, etc.

Supplied with the image signals CV_R, CV_G and CV_B flicker-corrected bythe flicker correction circuits 30R, 30G and 30B, respectively, and withthe flicker amplitude signals A_R, A_G and A_B adjusted by the flickeramplitude adjusters 60R, 60G and 60B, respectively, the correction phaseerror detectors 40R, 40G and 40B detect correction phase errors of theimage signals CV_R, CV_G and CV_B in the digitized image signals DV_R,DV_G and DV_, flicker-corrected image signals CV_R, CV_G and CV-B andflicker amplitude signals A_R, A_G and A_B to generate correction phaseerror signals E_R, E_G and E_B, and supply the generated correctionamplitude error signals E_R, E_G and E_B to the flicker correctioncircuits 30R, 30G and 30B, and correction amplitude error detectors 50R,50G and 50B, respectively.

Supplied with the image signals CV_R, CV_G and CV_B flicker-corrected bythe flicker correction circuits 30R, 30G and 30B, respectively, and withthe flicker amplitude signals A_R, A_G and A_B adjusted by the flickeramplitude adjusters 60R, 60G and 60B, respectively, the correctionamplitude error detectors 50R, 50G and 50B detect correction phaseerrors of the image signals CV_R, CV_G and CV_B in the digitized imagesignals DV_R, DV_G and DV_B, flicker-corrected image signals CV_R, CV_Gand CV_B and flicker amplitude signals A_R, A_G and A_B and correctionphase error signals E_R, E_G and E_B to generate correction amplitudeerror signals C_R, C_G and C_B, and supply the generated correctionamplitude error signals E_R, E_G and E_B to the flicker amplitudeadjusters 60R, 60G and 60B, respectively.

The flicker amplitude adjusters 60R, 60G and 60B generate flickeramplitude signals A_R, A_G and A_B from the correction amplitude errorsignals E_R, E_G and E_B, respectively, and supplies the generatedflicker amplitude signals A_R, A_G and A_B to the flicker correctioncircuits 30R, 30G and 30B, correction phase error detectors 40R, 40G and40B and correction amplitude error detectors 50R, 50G and 50B,respectively.

In the image pickup device 100, each of the flicker correction circuits30R, 30G and 30B uses a flicker correction circuit 30* constructed asshown in FIG. 6. It should be noted here that the asterisk (*) standsfor “R (red)”, “G (green)” and “B (blue)”.

The flicker correction circuit 30* includes an address calculator 31*supplied with a correction error signal E_* from the correction errordetector 40*, correction value calculator 32* supplied with an addressAD calculated by the address calculator 31*, multiplier 33* suppliedwith flicker correction data FD calculated by the correction valuecalculator 32* and flicker amplitude signal A_* generated by the flickeramplitude adjuster 60*, level adjuster 34* supplied with flickercorrection data FDA resulted from multiplication of the flickercorrection data FD by the flicker amplitude signal A_* in the multiplier33, and a low-pass filter (LPF) 35* and operational circuit 36*,supplied with an image signal DV_* digitized by the A-D converter 20*.The image signal DV_* digitized by the A-D converter 20* is supplied,via the low-pass filter (LPF) 35*, to the level adjuster 34* that willthen generate a flicker correction value CFD which is to be supplied tothe operational circuit 36*.

In the flicker correction circuit 30* constructed as above, the addresscalculator 31* calculates an address AD in ROMs (flicker memories 321and 322 which will further be described in detail later) included in thecorrection value calculator 32* on the basis of the correction errorsignal E_* supplied from the correction error detector 40*.

The address calculator 31* calculates the address of a present line bycalculating the address of a first line in a frame of interest from apower supply frequency and frame rate, and calculating an addressincrement at each advance by one line toward the address. Morespecifically, in case the power supply frequency is 50 Hz, frame rate is30 Hz and the number of vertical clocks of the image sensing device 10*is 1125 clk (these power supply frequency, frame rate and number ofclocks of the image sensing device 10* remain unchanged through thefollowing description), the period T between light and dark fringes of aflicker will contain 337.5 lines as given below by an equation 1:T=30 Hz×1125 clk/(50 Hz×2)=37.5 (clk)  (1)Also, the ROM in the system holds flicker data resulted from division ofone period by 512. At each advance by one line, the address in the ROMwill be incremented by about 1.51703 as given below by an equation 2:512/337.5=1.51703  (2)That is, on the assumption that the correction wave address on the firstline is zero (0), the address on the 100th line counted from the firstline will be 152 as given below by an equation 3:0+1.51703×100≈152  (3)

As shown in FIG. 7, the correction value calculator 32* includes flickermemories 321 and 322, multipliers 323 and 324 to multiply two types offlicker data FD1 and FD2 read from the flicker memories 321 and 322 bycoefficients α and α−1, respectively, and an adder 325 supplied withflicker data FD1_A and FD2_A multiplied by the coefficients α and α−1,respectively, by the multipliers 323 and 324, respectively. The twotypes of flicker data FD1 and FD2 will be read from the flicker memories321 and 322, respectively, according to the address AD calculated by theaddress calculator 31*.

The correction value calculator 32* calculates one flicker correctiondata FD by reading the two types of flicker data FD1 and FD2 from theflicker memories 321 and 322, respectively, on the basis of the addressAD calculated by the address calculator 31*, multiplying the flickerdata FD1 and FD2 by the coefficients α and α−1, respectively, by themultipliers 323 and 324, respectively, correspondingly to a frame rateand shutter speed, and adding the results together by the adder 325.

Note that the periodicity of the flicker data is utilized, thecorrection value calculator 32* is to hold a part of waveforms of theflicker data FD1 and FD2. Also, flicker data can appropriately becalculated even with any other memory than the ROM. In this embodiment,one flicker correction data FD corresponding to a waveform issynthesized by combining the two flicker data FD1 and FD2 together.However, three or more flicker data can be combined together tosynthesize various flicker correction data FD. The flicker correctiondata FD is updated once by a value depending upon each line per line.

Since the flicker varies in level correspondingly to the value of eachpixel, the level is adjusted for each by the use of the input imagesignal DV_*. However, a noise component in the image signal DV_* willinfluence the level adjustment.

On this account, in the flicker correction circuit 30* of the imagepickup device 100, the input image signal DV_* is passed through thelow-pass filter (LPF) 35* to remove the noise component from the imagesignal DV_* and the noise-free image signal DV_*′ is supplied to thelevel adjuster 34*. The level adjuster 34* can calculate a correctionvalue CFD for each pixel not influenced by the noise from the noise-freeimage data DV_*′ and flicker correction data FD calculated by thecorrection value calculator 32*.

Note that this embodiment is adapted so that the correction valuemonotonously increases correspondingly to a pixel value for there hasbeen observed a tendency that the flicker level also increases linearlycorrespondingly to a pixel value. Also, since no flicker is observedwhen the pixel value is extremely small or large, the embodiment isadapted to make a calculation taking this feature in account. However,the present invention is not limited to this embodiment.

In the flicker correction circuit 30*, the operational unit 35* adds thecorrection value CFD for each pixel to the image signal DV_* to providea corrected image signal CV_*.

The algorithm for detection of a correction error in the image pickupdevice 100 will be described below with reference to FIG. 8.

It is assumed here that a flicker of a certain frame image (image of then-th frame) could have been corrected to a correct level. Since theflicker is continuous from one frame to another, a flicker of a nextframe can be predicted. With the predicted next-frame flicker being keptat the same level as the flicker of an n-th frame, the flicker of a(n+1)th frame is added to the corrected image. A flicker image thusproduced is taken as “image A”. At the same time, with the level beingmade higher than that of the n-th frame, the flicker of the (n+1)thframe is added to the flicker-corrected image. A flicker image thusproduced is taken as “image B”. The image A is an image resulted fromaddition of the flicker component of the (n+1)th frame to an objectimage of the n-th frame, while the image B is resulted from addition ofthe flicker component of the (n+1)th frame whose flicker level has beenmade higher than that of the image A to the object image of the framen-th frame. When differences are calculated between the image of the(n+1)th frame actually supplied and images A and B, respectively, only amovement component of the object will appear as the differences becausethe next-frame flicker is faithfully reproduced in the image A. However,since the image B is made higher in level than the next-frame flicker,it contains two components, namely, the object movement and flickercomponent as the differences, that is, the predicted flicker componentmore approximate to the next-frame flicker will take a small value asthe difference. Further, there are generated two images, namely, animage having added thereto a flicker component of the (n+1)th frame,higher in level than the flicker of the n-th frame and an image havingadded thereto a flicker component of the (n+1)th frame, lower in levelthan the flicker of the n-th frame. When differences are calculatedbetween the two images and image of the (n+1)th frame, respectively, thedifference more approximate to the flicker level of the (n+1)th framewill take a small value. Therefore, it is possible to adjust the initiallevel of an arbitrary flicker to an appropriate flicker levelautomatically as the time elapses by making a comparison between thedifferences for each frame.

Each of the correction phase error detectors 40R, 40G and 40B andcorrection amplitude error detectors 50R, 50G and 50B is designedaccording to the above-mentioned algorithm.

In this image pickup device 100, each of the correction phase errordetectors 40R, 40G and 40B uses a correction phase error detector 40*constructed as shown in FIG. 9 according to the aforementionedalgorithm. It should be noted here that the asterisk (*) stands for “R(red)”, “G (green)” and “B (blue)”.

The correction phase error detector 40* includes flicker-added signalgenerators 41A and 41B supplied with an image signal CV_*flicker-corrected by the flicker correction circuit 30* and flickeramplitude signal A_* generated by the flicker amplitude adjuster 60*,line integrators 42A and 42B supplied with flicker-added signals FDV1and FDV2 generated by the flicker-added signal generators 41A and 41B,respectively, memories 43A and 43B supplied with line data LD11 and LD21integrated by the line integrators 42A and 42B, respectively, differencedetectors 44A and 44B supplied with line data LD12 and LD22 read fromthe memories 43A and 43B, respectively, line integrator 45 supplied withan image signal DV_* digitized by the A-D converter 20*, integrators 46Aand 46B supplied with difference data DD1 and DD2 detected by thedifference detectors 44A and 44B, respectively, comparator 47 suppliedwith integrated data ID1 and ID2 provided by the integrators 46A and46B, respectively, etc. Line data LD3 provided by the line integrator 45will be supplied to each of the difference detectors 44A and 44B, and acorrection error signal E_* provided as a comparison output from thecomparator 47 be supplied to each of the flicker-added signal generators41A and 41B.

Each of the flicker-added signal generators 41A and 41B includes addresscalculators 411A and 411B supplied with the correction error signal E_*supplied as a comparison output from the comparator 47, addressconverters 412A and 412B supplied with addresses AD11 and AD21calculated by the address calculators 411A and 411B, respectively,correction value calculators 413A and 413B supplied with addresses AD12and AD22 calculated by the address converters 412A and 412B,respectively, multipliers 414A and 414B supplied with a flickeramplitude signal A_* generated by the flicker amplitude adjuster 60*,level adjusters 415A and 415B supplied with flicker data FD1 and FD2multiplied by the flicker amplitude signal A_* in the multipliers 414Aand 414B, and low-pass filters (LPF) 416A and 416B and operational units417A and 417B, supplied with an image signal DV_* digitized by the A-Dconverter 20*. The image signal DV_* digitized by the A-D converter 20*will be supplied, via the low-pass filters (LPF) 416A and 416B,respectively, to the level adjusters 415A and 415B, and correctionvalues CFD1 and CFD2 generated by the level adjusters 415A and 415B,respectively, are supplied to the operational units 417A and 417B,respectively.

In the correction error detector 40* constructed as above, the addresscalculators 411A and 411B calculate addresses AD11 and AD21 in the ROMon the basis of the correction error signal E_*. The addresses to bethus calculated are resulted from shifting the top address of a flickerof a next frame in the positive- or negative-going direction. Theseaddresses are calculated as in the address calculator 31* in the flickercorrection circuit 30*. Also, the ROM included in the correction errordetector 40* is identical to that included in the flicker correctioncircuit 30*.

The address converters 412A and 412B convert the addresses AD11 and AD21calculated by the address calculators 411A and 411B, respectively, intoaddresses AD12 and AD22, respectively, from which flickers of a nextframe can be reproduced. That is, they convert the addresses AD1 and AD2into addresses having opposite phases. The addresses AD12 and AD22converted by the address converters 412A and 412B, respectively, areresulted from prediction of flickers of the next frame, but not intendedfor correction of the flickers.

The correction value calculators 413A and 413B calculate flicker dataFD1 and FD2 on the basis of the addresses AD12 and AD22, respectively,converted by the address converters 412A and 412B, respectively. Theflicker data FD1 and FD2 are also determined per line as in the flickercorrection circuit 30*. The correction value calculators 413A and 413Bare similarly constructed to the correction value calculator 32*included in the flicker correction circuit 30*.

As in the flicker correction circuit 30*, in the flicker-added signalgenerators 41A and 41B of the correction phase error detector 40*, theimage signal DV_* is passed through the low-pass filters (LPF) 416A and416B to remove noises from the image signal DV_*, the noise-free imagesignal DV_*′ is supplied to the level adjusters 415A and 415B. The leveladjusters 415A and 415B calculate correction values CFD1 and CFD2 foreach pixel not influenced by the noises on the basis of the noise-freeimage signal DV_*′ and flicker data FD1 and FD2 calculated by themultipliers 414A and 414B, respectively.

The level adjusters 415A and 415B are constructed like the leveladjuster 34* included in the flicker correction circuit 30*.

The operational units 417A and 417B add the correction values CFD1 andCFD2 for each pixel to the flicker-corrected image signal CV_* togenerate flicker-added signals FDV1 and FDV2 for a next frame. Theoperational units 417A and 417B are also constructed like theoperational unit 36 included in the flicker correction circuit 30*.

The line integrators 42A and 42B calculate line data LD1 and LD21 byintegrating certain segments of the flicker-added signals FDV1 and FDV2of the next frame, respectively. The “segment” may be of an arbitraryvalue as a horizontal size so far as it is within an image acquiredhorizontally. With a larger segment, a correction error can be detectedwith a higher accuracy. The vertical size of the segment may be anintegral multiple of the cycle of the light and dark fringes of aflicker within one screen. More specifically, the segment may be given asize of 1000 horizontal pixels by 675 vertical pixels (=337.5×2),namely, of 1000×675 pixels.

The line data LD11 and LD21 calculated by the line integrators 42A and42B are stored in the memories 43A and 43B, respectively, until theimage signal DV_* of a next frame is supplied. When the image signalDV_* of the next frame is supplied, the line integrator 45 makes lineintegration of the same segments as those of the flicker-added signalsFDV1 and FDV2 which have been integrated to calculate the line data LD3.

The line data LD12 and LD22 stored in the memories 43A and 43B,respectively, and line data LD3 of the next-frame image signal DV_*corresponding to the lines of the line data LD12 and LD22 are suppliedto the difference detectors 44A and 44B to provide difference data DD1and DD2.

The integrators 46A and 46B provide integrated data ID1 and ID2,respectively, by integrating the two difference data DD1 and DD2,respectively.

Then, the comparator 47 judges, by making a comparison in size betweenthese integrated data ID1 and ID2, in which direction an address isshifted for prediction of a correct flicker, positive- ornegative-going. For example, in case the integrated data ID* obtainedwith the address shifted in the positive-going direction is smaller thanthe integrated data ID* obtained with the address shifted in thenegative-going direction, a correction phase error signal E_* isoutputted to shift the address in the positive-going direction.

The correction error is minimized by supplying the correction phaseerror signal E_* to the address calculator 31 of the flicker correctioncircuit 30* and address calculators 511A and 511B of the correctionamplitude error detector 50* to shift the address in a correct directiontoward a flicker.

Also in this image pickup device 100, each of the correction amplitudeerror detectors 50R, 50G and 50B uses the correction amplitude errordetector 60* constructed as shown in FIG. 10 according to theaforementioned algorithm. It should be noted here that the asterisk (*)stands for “R (red)”, “G (green)” and “B (blue)”.

The correction amplitude error detector 50* includes flicker-addedsignal generators 51A and 51B supplied with an image signal CV_*flicker-corrected by the flicker correction circuit 30* and flickeramplitude signal A_* generated by the flicker amplitude adjuster 60*,line integrators 52A and 52B supplied with flicker-added signals FDV13and FDV32 generated by the flicker-added signal generators 51A and 51B,respectively, memories 53A and 53B supplied with line data LD31 and LD32integrated by the line integrators 52A and 52B, respectively, differencedetectors 54A and 54B supplied with line data LD31 and LD32 read fromthe memories 53A and 53B, respectively, line integrator 55 supplied withan image signal DV_* digitized by the A-D converter 20*, integrators 56Aand 56B supplied with difference data DD31 and DD32 detected by thedifference detectors 54A and 54B, respectively, comparator 57 suppliedwith integrated data ID31 and ID32 provided by the integrators 56A and56B, respectively, etc. Line data LD3 provided by the line integrator 55will be supplied to each of the difference detectors 54A and 54B.

The flicker-added signal generators 51A and 51B includes addresscalculators 511A and 511B, respectively, supplied with the correctionphase error signal E_* provided as a comparison output from thecomparator 47 in the correction phase error detector 40*, addressconverters 512A and 512B supplied with an address AD31 calculated by theaddress calculators 511A and 511B, respectively, correction valuecalculators 513A and 513B supplied with an address AD32 calculated bythe address converters 512A and 512B, respectively, amplitude amplifier514A and amplitude attenuator 514B, supplied with the flicker amplitudesignal A_* generated by the flicker amplitude adjuster 60*, multipliers515A and 515B supplied with flicker data FD31 and CFD31 calculated bythe correction value calculators 513A and 513B, respectively, leveladjusters 516A and 516B supplied with flicker data CFD1 and CFD2multiplied by the flicker amplitude signal A_* in the multipliers 515Aand 515B, and low-pass filters (LPF) 517A and 517B and operational units518A and 518B, supplied with an image signal DV_* digitized by the A-Dconverter 20*. Amplitude signals AM31 and AM32 resulted from controllingof the flicker amplitude signal A_* in the amplitude amplifier 515A andamplitude attenuator 514B are supplied to the multipliers 515A and 515B,respectively, the image signal DV_* digitized by the A-D converter 20*will be supplied, via the low-pass filters (LPF) 517A and 517B,respectively, to the level adjusters 516A and 516B, and correctionvalues CFD31 and CFD32 generated by the level adjusters 516A and 516B,respectively, are supplied to the operational units 518A and 518B,respectively.

In the flicker-added signal generators 51A and 51B of the correctionamplitude error detector 50* constructed as above, the addresscalculators 511A and 511B calculate the address AD31 in the ROM on thebasis of the correction error signal E_*. The address to be thuscalculated is the top address of a flicker component of a next frame.This address is calculated as in the address calculator 31* in theflicker correction circuit 30*. Also, the ROM included in the correctionamplitude error detector 50* is identical to that included in theflicker correction circuit 30*.

The address converters 512A and 512B convert the address AD31 calculatedby the address calculators 511A and 511B into an address from which aflicker of a next frame can be reproduced. More specifically, theyconvert the phase of the address AD31 into an address AD32 opposite inphase to the address AD31. The addresses AD32 converted by the addressconverters 512A and 512B is resulted from prediction of flickers of thenext frame, but not intended for correction of the flickers.

The correction value calculators 513A and 513B calculate flicker dataFD31 on the basis of the address AD32 converted by the addressconverters 512A and 512B. The flicker data FD31 is also determined perline as in the flicker correction circuit 30*. The correction valuecalculators 513A and 513B are similarly constructed to the correctionvalue calculator 32* included in the flicker correction circuit 30*.

Next, the amplitude amplifier 514A calculates an amplitude signal AM31resulted from amplification of the amplitude signal A_*. At the sametime, the amplitude attenuator 514B attenuates the supplied amplitudesignal A_* to calculate an amplitude signal AM32. The multipliers 515Aand 515B multiply the calculated amplitude signals AM31 and AM32 by theflicker data FD31 to provide flicker correction data FDA31 having aflicker component thereof amplified and flicker correction data FDA32having a flicker component thereof attenuated.

Also in the flicker-added signal generators 51A and 51B of thecorrection amplitude error detector 50*, the image signal DV_* is passedthrough the low-pass filters (LPF) 517A and 517B as in the flickercorrection circuit 30* to remove noises from the image signal DV_*, andthe noise-free image signal DV_*′ is supplied to the level adjusters516A and 516B. The level adjusters 516A and 516B calculate correctionvalues CFD31 and CFD32 for each pixel from the noise-free image signalDV_*′ and flicker data FDA31 and FDA32 calculated by the correctionvalue calculators 515A and 515B, respectively.

The level adjusters 516A and 516B are constructed like the leveladjuster 34* included in the flicker correction circuit 30*.

The operational units 518A and 518B add the correction values CFD31 andCFD32 for each pixel to the flicker-corrected image signal CV_* togenerate flicker-added signals FDV31 and FDV32 of a next frame. Theoperational units 518A and 518B are similarly constructed to theoperational unit 36 included in the flicker correction circuit 30*.

Then, the flicker-added signals FDV31 and FDV32 of a next frame,supplied from the operational units 518A and 518B, respectively, areprocessed as in the correction phase error detector 40*.

That is, the line integrators 52A and 52B integrate segments of the nextframe where the flicker-added signals FDV31 and FDV32 exist to calculateline data LD31 and LD32.

The line data LD31 and LD32 calculated by the line integrators 52A and52B are stored in the memories 53A and 53B, respectively, until theimage signal DV_* of a next frame is supplied. When the image signalDV_* of the next frame is supplied, the line integrator 55 makes lineintegration of the same segments as those of the flicker-added signalsFDV31 and FDV32 which have been integrated to calculate the line dataLD3.

The line data LD31 and LD32 stored in the memories 53A and 53B,respectively, and line data LD3 of the next-frame image signal DV_*corresponding to the lines of the line data LD31 and LD32 are suppliedto the difference detectors 54A and 54B to provide difference data DD31and DD32.

The integrators 56A and 56B provide integrated data ID31 and ID32,respectively, by integrating the two difference data DD31 and DD32,respectively.

Then, the comparator 57 makes a comparison in size between theseintegrated data ID31 and ID32 to provide a comparison signal C*indicative of whether the amplitude of the next-frame flicker should beamplified or attenuated.

The comparison signal C_* provided in the correction amplitude errordetector 50* is supplied to the flicker amplitude adjuster 60*. As shownin FIG. 11, the flicker amplitude adjuster 60* includes a comparator 61and amplitude increasing/decreasing unit 62 and always varies theflicker amplitude on the basis of the comparison signal C_*. The flickeramplitude adjuster 60* functions to vary the flicker amplitude in adirection in which the comparison signal C_* supplied from thecorrection amplitude error detector 50* will be smaller. It should benoted here that a smaller comparison signal C_* means that aprediction-based flicker image is more approximate to an actual imageformed by the image sensing device 10*. Namely, examining, by thecomparator 61, the relation in size of the comparison signal C_* betweenthe frames permits to judge whether the flicker level has been predictedaccurately. For example, in case the comparison signal C_* is largerthan that of a preceding frame when the flicker amplitude has beenincreased, it can be determined that the flicker amplitude has beenpredicted inaccurately. In this case, the flicker amplitude is to bedecreased by the amplitude increasing/decreasing unit 62. On thecontrary, in case the comparison signal C_* is smaller than that of thepreceding frame, it can be determined that the flicker amplitude hasbeen predicted accurately. In this case, the flicker amplitude is to beincreased by the amplitude increasing/decreasing unit 62. With theseoperations, the amplitude signal A_* is provided as an output.

Thus, by varying, according to the aforementioned correction errordetection algorithm, the level in a direction in which the integratedvalue of the two difference data DD31 and DD32 will be smaller, it ispossible to vary the correction level to an optimum one as the timeelapses.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

1. A flicker correction method of a flicker correction device comprisingthe steps of: predicting, from an image of a present flicker-correctedframe, a flicker of an image of a next frame to generate first andsecond types of flicker images having flickers different in level fromeach other added thereto; detecting a flicker amplitude componentthrough comparison between (a) the generated first type of flicker imageand an image of an input next frame and (b) the generated second type offlicker image and the image of the input next frame; generating aflicker amplitude correction value on the basis of the detected flickeramplitude component; and making flicker amplitude correction by addingthe generated flicker amplitude correction value to an input image frameby frame.
 2. A flicker correction device comprising: a flickercorrecting means for making flicker amplitude correction by adding aflicker amplitude correction signal to an input image signal; and aflicker correction signal generating means for predicting a flicker ofan image of a next frame from the image signal having beenflicker-corrected by the flicker correcting means and an image signalnot yet flicker-corrected to generate first and second types of flickerimages to which flickers different in level from each other are addedthereto, for detecting a flicker amplitude component through comparisonbetween (a) the generated first type of flicker image and an image of aninput next frame and (b) the generated second type of flicker image andthe image of the input next frame and for generating a flicker amplitudecorrection value on the basis of the detected flicker amplitudecomponent, the flicker correcting means making flicker amplitudecorrection by adding the flicker amplitude correction signal generatedframe by frame by the flicker correction signal generating means to theinput image signal.
 3. The device according to claim 2, wherein theflicker correction signal generating means comprises: a correction phaseerror detecting means for detecting a correction phase error bypredicting a flicker of an image signal of a next frame from the imagesignal flicker-corrected by the flicker correcting means and an imagesignal not yet flicker-corrected to generate first and second types offlicker image signals having flickers different in level to each otheradded thereto and making a comparison between (a) the generated firsttype of flicker image signals and an image signal of an input next frameand (b) the generated second type of flicker image signals and the imagesignal of the input next frame; a correction amplitude error detectingmeans for detecting a correction amplitude error by predicting a flickerof an image signal of a next frame from the image signalflicker-corrected by the flicker correcting means and an image signalnot yet flicker-corrected to generate first and second types of flickerimage signals having flickers different in level to each other addedthereto and making a comparison between (a) the generated first type offlicker image signals and an image signal of an input next frame and (b)the generated second type of flicker image signals and the image signalof the input next frame; and a flicker amplitude adjusting means forgenerating an amplitude signal according to which the amplitude of aflicker correction signal is controlled in a direction in which thecorrection amplitude error detected by the correction amplitude errordetecting means will be smaller, the flicker correction signal beinggenerated based on the correction phase error signal generated by thecorrection phase error detecting means and amplitude signal generated bythe flicker amplitude adjusting means.
 4. An image pickup deviceincluding a flicker correction device to make flicker correction byadding a flicker correction signal to an image signal captured by theimage pickup device, the flicker correction device comprising: a flickercorrecting means for making flicker amplitude correction by adding aflicker amplitude correction signal to an input image signal; and aflicker correction signal generating means for predicting a flicker ofan image of a next frame from the image signal having beenflicker-corrected by the flicker correcting means and an image signalnot yet flicker-corrected to generate first and second types of flickerimages to which flickers different in level from each other are addedthereto, for detecting a flicker amplitude component through comparisonbetween (a) the generated first type of flicker image and an image of aninput next frame and (b) the generated second type of flicker image andthe image of the input next frame and for generating a flicker amplitudecorrection value on the basis of the detected flicker amplitudecomponent, the flicker correcting means making flicker amplitudecorrection by adding the flicker amplitude correction signal generatedframe by frame by the flicker correction signal generating means to theinput image signal.
 5. A flicker correction device comprising: a flickercorrection circuit for making flicker amplitude correction by adding aflicker amplitude correction signal to an input image signal; and aflicker correction signal generator for predicting a flicker of an imageof a next frame from the image signal having been flicker-corrected bythe flicker correction circuit and an image signal not yetflicker-corrected to generate first and second types of flicker imagesto which flickers different in level from each other are added thereto,for detecting a flicker amplitude component through comparison between(a) the generated first type of flicker image and an image of an inputnext frame and (b) the generated second type of flicker image and theimage of the input next frame and for generating a flicker amplitudecorrection value on the basis of the detected flicker amplitudecomponent, the flicker correction circuit making flicker amplitudecorrection by adding the flicker amplitude correction signal generatedframe by frame by the flicker correction signal generator to the inputimage signal.
 6. An image pickup device including a flicker correctiondevice to make flicker correction by adding a flicker correction signalto an image signal captured by the image pickup device, the flickercorrection device comprising: a flicker correction circuit for makingflicker amplitude correction by adding a flicker amplitude correctionsignal to an input image signal; and a flicker correction signalgenerator for predicting a flicker of an image of a next frame from theimage signal having been flicker-corrected by the flicker correctioncircuit and an image signal not yet flicker-corrected to generate firstand second types of flicker images to which flickers different in levelfrom each other are added thereto, for detecting a flicker amplitudecomponent through comparison between (a) the generated first type offlicker image and an image of an input next frame and (b) the generatedsecond type of flicker image and the image of the input next frame andfor generating a flicker amplitude correction value on the basis of thedetected flicker amplitude component, the flicker correction circuitmaking flicker amplitude correction by adding the flicker amplitudecorrection signal generated frame by frame by the flicker correctionsignal generator to make flicker correction to the input image signal.7. The method according to claim 1, wherein the flicker amplitudecomponent varies in a direction in which the comparison is smaller. 8.The method according to claim 1, wherein the flicker amplitude componentindicates whether an amplitude of a next frame flicker is amplified orattenuated.